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ABSTRACT 



The development of standardized performance measurements, 
the widespread use of such measures in Medicaid, and the planned uses of such 
measures in the Child Health Insurance Program (CHIP) provide significant 
opportunities for child advocates to improve child health services. This 
issue brief describes the growth of managed care organizations, presents 
characteristics of performance measurement, and explores the conceptual 
framework underlying performance measurement in managed care. The brief 
describes the major approach to performance measurement currently in use. 

Also described is the widespread and growing use of performance measurement 
tools by state governments to assess quality in Medicaid and CHIP plans. 
Finally, the brief discusses how child advocates can use performance 
measurement information to help improve CHIP and Medicaid services for 
children. (KB) 
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Introduction 1 

C hild advocates have long understood the importance 
of good data in improving conditions for children. 
Data on eligibility and enrollments, in the Child 
Health Insurance Program (CHIP) and Medicaid haver 
been high priority topics among child advocates for 1 
some time. Child advocates have long been interested in V 
data on quality and service as well. Once children are;';.:'/;' '••/;' 
enrolled in CHIP or Medicaid, do they receive immuniza- 
tions? Well-child care? Are providers available without ' 
delay? Do health plans give parents accurate and compre- 
hensible information? 

The emergence of managed care as a major form of health 
care delivery in the publicly funded child health environ- 
ment may result in valuable new sources of credible data 
on health plan quality and performance, and give child 
advocates important new tools in their advocacy work. The 
growth of managed care has been accompanied by the devel- 
opment of a new type of performance measurement. This 
type of measurement utilizes precise, standardized methods, 
includes rigorous specifications for data elements, and pro- 
duces comparable results. Based on the concept that man- 
aged care organizations 1 can and should be held accountable 
for providing services to their enrolled members, this 
approach to performance measurement is used widely for 
managed care organizations (MCOs) providing services to 
employees and their families, and is mandated by the federal 
Health Care Financing Administration (HCFA) for use by all 
Medicare managed care plans. States are also adopting this 
approach to performance measurement for use with their 
contracted Medicaid managed care organizations, and for 
^ MCOs delivering services to CHIP enrollees. 




•he development of standardized performance 
measurement, its widespread use in Medicaid, 
and its planned uses in CHIP programs around the 
country provide significant opportunities for child 
ddvocates to improve child health services. Performance 
measurement can increase the accountability of MCOs 
providing health care services to children in CHIP and 
Medicaid programs. It can drive programs of quality 
improvement in both the delivery of health care services 
arid in consumer service. There are unique opportuni- 
ties for child advocates in the states to play critical roles 
in shaping how performance measurement in CHIP 
and Medicaid is structured, how it is used, and whether 
it leads to improved health care for children. 



This Issue Brief explores the conceptual framework underlying 
performance measurement in managed care. It describes the 
major approaches to performance measurement currently in 
use. It describes the widespread and growing use of perform- 
ance measurement tools by state governments to assess quality 
in Medicaid and CHIP plans. Finally, it discusses how child 
advocates can use performance measurement information to 
help improve CHIP and Medicaid services for children. 
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Managed Care Increasing 

A ccording to HCFA’s most current 
data, as of June 30, 1998, Medicaid 
enrollment in comprehensive man- 
aged health care organizations 
(MCOs) and Medicaid-only MCOs 
totaled 11.9 million of a total of 30.9 mil- 
lion Medicaid beneficiaries. Since children 
ages 0-20 totaled 53.5 percent of Medicaid 
enrollees as of federal fiscal year 1997, we 
can estimate that approximately 6.3 million 
children are enrolled in Medicaid MCOs. 
This number is expected to continue to 
increase, since enrollment in Medicaid 
managed care is continuing to grow, 2 
According to available descriptions of state 
CHIP plan applications, approximately 
two-thirds of the states indicated in those 
applications that they intended to use the 
same service delivery system for CHIP and 
for Medicaid, in whole or in part. 3 

Along with this increased use of MCOs as 
the vehicle for delivering health care servic- 
es to children in publicly funded health 
care programs has come the increasingly 
widespread use of MCO performance 
measurement tools. Use of these new per- 
formance measurement tools offers many 
opportunities to child advocates. 

These performance measurement tools are 
standardized, use precise definitions, have 
been tested for reliability and validity, and 
have been tested in the field. As a result, 
these tools can be used to accurately and 
reliably compare MCO performance in the 
actual delivery of care across plans, 
between states, and even nationally. Using 
these standard “measuring sticks,” MCO 
performance can be compared year to year. 
Quality improvement programs can be 
applied with new rigor, since the same pre- 
cise, standardized performance measure 
can be used to measure the baseline MCO 
performance and the level of MCO per- 
formance after the quality improvement 
intervention has been developed and 
implemented. Performance measurement 
results can be - and sometimes are - widely 
used in reports to legislatures, state offi- 
cials, and the public. They can be used to 
develop benchmarks. Because of their pre- 
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cise and standardized nature, they can also 
be used for report cards comparing plan 
results for use by consumers and others. 
All told, they offer child advocates impor- 
tant new opportunities to ensure that, after 
children are enrolled, they receive quality 
health care services. 

Performance Measurement 
Characteristics 



P 



erformance measurement in health 
care is not unique to managed care. 
However, the rapid growth of 
managed care organizations 
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and the shift away from 
fee-for-service (FFS) medicine 
created a new demand for 
accurate information on 
how managed care plans 
performed, and for meas- 
urement which produced 
results that could be 
compared across health 
plans, across geographic 
locations, and over time. 

In effect, the emerging 
dominance of MCOs both 
allowed and required the 
development of performance 
measurement techniques 
peculiar to managed care. 4 



Managed care performance measure- 
ment is premised on the concept that man- 
aged care plans can and should be held 
accountable for providing quality care to 
their enrolled members. The nature of 
managed care performance measurement 
was shaped by the distinctive characteris- 
tics of MCOs. 

An MCO has a discrete, identifiable set of 
enrolled members . It is responsible for provid- 
ing a defined set of services and benefits to 
them . The health plan automatically 
receives a prepaid set amount each month 
for each member enrolled - the “capitation” 
payment. Whether or not benefits or servic- 
es have been provided to some or all 
enrolled members, and whether or not ben- 
efits and services are utilized at, below, or 
above the levels and/or costs projected by 
the health plan, the health plan is responsi- 
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ble for providing the defined services and 
benefits for all enrollees, and for living with- 
in its budget. If the MCO’s expenses exceed 
its revenues, it takes a loss. In FFS medi- 
cine, on the other hand, each individual 
seeks out providers, the providers adminis- 
ter whatever care they deem necessary or 
desirable, the providers bill the insurance 
company, and the insurance company pays. 
(Over time co-pays and deductibles were 
introduced to try to restrain utilization, but 
the basic concept remained unchanged.) 

Only individuals who present them- 
selves receive services; providers 
have no responsibility for out- 






the development of 



reach. There is no health care 
“budget.” Neither the 
provider nor the insurance 
company is accountable 
to the patient or the pur- 
chaser (typically a pri- 
vate sector employer or 
a public entity such as a 
state Medicaid agency). 






performance measurement^ 



'"•techniques peculiar to 
managed care.l 




Further, in managed 
care, the contract between 
the managed care plan and 
the purchaser describes the 
services that the plan is obli- 
gated to provide. Because the 
capitation payment is based on 
calculations concerning the cost 
of these services and the extent to 
which they will be utilized by the projected 
enrolled patient population, plans are not 
obligated to provide services beyond the 
contractually specified limit. In FFS, there is 
no contractual description of services. As 
noted above, there is no budget. If spending 
exceeds that projected in a year, the insur- 
ance company raises rates for the next 

Among the defined services provided by 
managed care plans are preventive or well- 
care benefits - immunizations, routine 
physicals, screenings, well-child care - typi- 
cally provided at low or even no cost to the 
enrolled member. The world of FFS was a 
world of sick care, not health care. So- 
called “health care” insurance covered only 
care for treatment when the patient was 
sick or had a condition which needed diag- 
nosis. Managed care pioneered in the pro- 
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vision of such preventive and well-care 
benefits. Until recendy, the typical FFS 
insurance plan did not provide reimburse- 
ment for such services as preventive 
screenings or routine physicals, and cover- 
age of such preventive and well-care servic- 
es in the FFS world is still incomplete. 

In managed care, the managed care organi- 
zation contracts with a specified list of 
providers - doctors, psychologists, pharma- 
cists, RNs, physical therapists, and more - 
and with a specified list of facilities - hospi- 
tals, clinics, and others - to provide services 
to the members enrolled in that managed care 
plan. Those providers and facilities agree 
to provide services to the plan’s enrolled 
members under terms and conditions 
specified in the contractual agreement 
with the plan. The fact that such a con- 
tract exists means that provider behavior 
can be influenced by the managed care 
plan through a variety of means, including 
provider education, utilization review con- 
ducted by the plan, comparison of per- 
formance with specified clinical standards 
or with the performance of specialist peers 
within the plan, and financial arrange- 
ments designed to encourage or discour- 
age specified types of provider behavior. 

As a result, plans have the opportunity to 
affect provider behavior in a fashion which 
is lacking in fee-for-service medicine. 

MCO ability to influence providers is a key 
factor underlying the demand for MCO 
accountability. 

Additionally, managed care incorporates 
the concept that enrollment over a reason- 
able period of time - “ continuous enrollment” 

- is required before a health plan can be held 
accountable for delivery of services to mem- 
bers. This is especially germane for preven- 
tive and well-care services such as immu- 
nizations, screenings, and other well-care 
services that are provided to plan members 
who are, by definition, well. Health plans 
are expected to conduct member outreach 
and provider education, among other 
things, to ensure high levels of preventive 
and well-child care; such health plan activi- 
ties require time to accomplish. Building 
on these characteristics, new types of 
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health care performance measurement 
have developed and become established. 

It’s a Simple Concept 

M anaged care performance measurement 
can be readily expressed as a simple 
fraction A/B=C. 

For example, one widely used performance 
measure, the HEDIS measure of childhood 
immunization status (see section on 
HEDIS below), is designed as follows: 

A/B = C, where: 

B A is the number of eligible persons 
(enrolled children who turned 2 during 
the reporting year) enrolled for the req- 
uisite length of time (a year prior to 
their 2nd birthday) who actually 
received the service or benefit in ques- 
tion (the clinically desired vaccinations 
administered in the approved time 
sequence); 

B B is the total number of eligible per- 
sons enrolled for the requisite length of 
time, regardless of whether they 
received the service (or even saw a 
provider); and 

■ C is the resulting rate of two year olds 
with the appropriate immunizations. 

Performance measures describe how an 
MCO performs in providing a benefit or 
service where there is a known and clinical- 
ly “desired” level, such as childhood immu- 
nizations. They describe MCO perform- 
ance in areas in which there is not enough 
information to set a target level but where 
wide variation from the norm is suspect, 
such as utilization of inpatient services. 
They also reveal how an MCO rates in the 
views of its enrolled members on such 
matters as access to providers, referrals to 
specialty care, or thoroughness of care. 

HEDIS® 

erformance measurement in man- 
aged care is dominated by the meas- 
urement set with the unlikely 
acronym of “HEDIS” (pronounced 
hee-dis, which stands for “Health 
[Plan] Employer Data Information Set” ). 



Use of the HEDIS performance measure- 
ment set is common among employer- 
based MCOs and is required by HCFA of 
Medicare + Choice MCOs (managed care 
organizations available to Medicare benefi- 
ciaries in many areas). Performance meas- 
urement is required of Medicaid managed 
care plans by the Balanced Budget Act of 
1997 (BBA), is written into HCFA’s Quality 
Improvement System for Managed Care 
standards (QISMC), and is included in the 
BBA legislation passed by Congress author- 
izing the CHIP program. 

While the BBA does not require that 
HEDIS be used in Medicaid managed care, 
the American Public Human Services 
Association’s (APHSA) study of HEDIS use 
cited earlier found widespread use of 
HEDIS performance measures to assess 
quality in state Medicaid programs. 
Twenty-eight states used or were planning 
to use HEDIS measures to monitor 
Medicaid managed care plan performance 
as of July, 1998.5 A number of other states 
either already use other performance 
measurement techniques, or are consider- 
ing what approach to adopt. A review of 
the intended performance measures listed 
in state CHIP plans submitted by 47 states 
and the District of Columbia found that all 
but four states reported plans to use all or 
part of the HEDIS performance measure- 
ment set to monitor the performance of 
their state CHIP programs. 6 

The HEDIS performance measurement data 
set is the product of over a decade of efforts 
begun by a group of health plans and large 
employers that wanted to differentiate 
among the quality of care provided by 
MCOs. It was continued by the National 
Committee for Quality Assurance (NCQA). 7 
HEDIS is the most widely known and 
broadly used approach to MCO perform- 
ance measurement at present. 

HEDIS measures are highly precise, stan- 
dardized, and prescribed. They are tested 
for validity, reliability, and feasibility. In 
general, HEDIS measures are calculated on 
the basis of data in medical records or in 
MCO administrative data bases. Allowable 
medical service codes are specified. 




The exact time periods during which 
services must have been administered 
are delineated. Acceptable methods of 
documentation are described. A specific 
methodology for selecting medical records 
(if they are used) is provided and must 
be followed to ensure that they are statis- 
tically representative. 

An illustrative description of a HED1S 
measure, that of Adolescent Immunization 
Status, is below. In HEDIS Volume 2, 
Technical Specifications, this description is 
followed by four and one-half pages of defi- 
nitions and specifications. 

Description: 

The percentage of Medicaid and commercially 
enrolled adolescents whose 13th birthday was in 
the reporting year, who were continuously 
enrolled for 12 months immediately preceding 
their 13th [with no more than one break of no 
more than 45 days] and who were identified as 
having had a second dose of MMR or a seropos- 
itive test result for measles , mumps and/or 
rubella and either three hepatitis B ora seropos- 
itive test result for hepatitis B and one VZV, a 
seropositive test result for VZV or evidence of 
chicken pox by the member's 13th birthday 

This precision - a precision reinforced by 
the HEDIS Compliance Audit developed 
by NCQA and conceptually akin to a finan- 
cial audit - means that collection of the 
appropriate data can be a challenge, espe- 
cially for health plans that do not have 
sophisticated management information sys- 
tems. It also means that HEDIS results, 
especially when audited as required by 
HCFA for Medicare+Choice MCOs and by 
private sector purchasers for many com- 
mercial MCOs, are precise measuring tools. 
They can be used to identify trends in plan 
performance over time, to compare per- 
formance across plans, states or regions, to 
compare plan or state results against 
regional or national benchmarks, and to 
set quality improvement goals against 
which progress can be precisely measured. 
NCQA continues to update and modify its 
HEDIS measurement set annually, and has 
measurement development field testing 
underway on many measures, in particular 
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those dealing with health outcomes and 
follow-up after acute illness. 

It is important to note that several states 
are pioneering in the application of 
HEDIS measures to their Primary Care 
Case Management (PCCM) programs in 
addition to their Medicaid MCOs. 
Approximately lour million Medicaid ben- 
eficiaries are enrolled in PCCM programs. 
While technically considered “managed 
care” in the Medicaid environment, PCCM 
programs do not have the tightly man- 
aged systems characteristic of MCOs. 
Massachusetts has collected HEDIS data 
for its PCCM program for several years. 
Colorado has also begun collecting 
HEDIS results for its PCCM program as 
well as for its Medicaid MCOs. 



The strengths of HEDIS are also its limits. 
HEDIS depends on accurate data. 
Information has its price: gathering data 
and ensuring its accuracy adds costs, both 
for health plans and purchasers. Additionally, 
the methodology is unforgiving. If an 
immunization is administered a day late, it 
cannot be counted in the plan’s immuniza- 
tion rate; if a parent says that an immuniza- 
tion has been received but cannot provide 
acceptable documentation, no immuniza- 
tion can be reported. 

Because of the complexity and cost of col- 
lecting and reporting HEDIS data, there are 
also limits to the number of measures which 
MCOs can be expected to report. HEDIS 
does not fully map the core elements of the 
EPSDT requirements, although it does 



Table 1: HEDIS Measures Used by 13+ State. Medicaid Agencies}. 



Childhood immunization status 24 states 

Well-child visits in the first 15 months 24 states 

Adolescent immunization status 14 states 

Well-child visits in the 3rd, 4th, 5th, and 6th years of life 19 states 

Prenatal care in the first trimester 22 states 

Adolescent well-care visits 17 states 

Incidence of low birth-weight babies 13 states 

Discharge and average length of stay-maternity care 22 states 

Children s access to primary care providers 16 states 

C-section and vaginal birth after C-section 18 states 

Availability of primary care providers 15 states 

Births and average length of stay, newborns 18 states 

Initiation of prenatal care 13 states 



Table: X:. HEDIS Measures Slated for Use. iivCHIPPlans* 



Well Child Care 42 states 

Emergency Room Utilization 31 states 

Adolescent Well Visits 4 states 

Mental Health Services/Utilization 35 states 

Immunizations 46 states 



Dental Care 34 of the 44 states offering dental benefits 
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directly report some important components. 
There is competition among various con- 
stituencies as to what measures should have 
priority in development and in reporting. 
HEDIS describes “what is the case” with 
great precision, but sheds no light on under- 
lying causes or explanations of variation. 
Finally, HEDIS’ precision and largely clinical 
orientation limits its audience. 

HEDIS , Medicaid , and CHIP 

H EDIS is widely used in the Medicaid 
environment. As noted, at least twen- 
ty-eight states require Medicaid 
MCOs to report HEDIS results. In 
some states, the requirements have 
been in place for some years. Many states 
use these data for public reports or report 
cards, based in whole or in part on the 
results. The APHSA is developing the first 
national database of HEDIS reports submit- 
ted by MCOs serving Medicaid beneficiaries. 

Some measures are in especially wide- 
spread use. Table 1 (page 4) lists the 
HEDIS measures which at least thirteen 
state Medicaid agencies require their con- 
tracted Medicaid MCOs to collect and 
report, listing the measure and the number 
of states requiring it on the right. 8 

While performance measurement in CHIP 
is in its infancy, Congressional language 
establishing the state CHIP programs 
required that each state “assure the quality 
and appropriateness of care” in its CHIP 
program. It also mandated that each state 
plan application “describe how perform- 
ance under the plan will be measured 
through objective, independendy verifiable 
means and compared against performance 
goals in order to determine the state’s per- 
formance.” Review of state CHIP plan 
applications show that many states plan to 
use HEDIS to satisfy these requirements in 
whole or in part. The HRSA/AHCPR 9 
analysis of states’ planned use of HEDIS 
measures in CHIP plans found that HEDIS 
measures are included in large numbers of 
state CHIP proposals. Table 2 (page 4) lists 
the major types of HEDIS measures includ- 
ed in state CHIP plan proposals, with the 
number of states indicated on the right. 
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Consumer Assessment of 
Health Plans (CAHPS) 

EDIS established the pattern for 
MCO performance measurement. 
However, the original set of HEDIS 
measures paid little attention to the 
perceptions of enrolled MCO mem- 
bers themselves. The HEDIS measurement 
set focused on measures of services with 
clinically based desired levels of perform- 
ance; measures of member access to servic- 
es; and measures of the utilization of vari- 
ous inpatient and outpatient services, few 
of which had clinically based target levels 
of performance. There were several smaller 
categories, largely descriptive in nature. 
While a member satisfaction measure was 
included, it was designed for the MCO 
enrollees receiving services through 
employer-provided MCOs, and received rel- 
atively little attention. 

As interest in MCO performance grew, 
there was an increasing demand for reli- 
able, standardized and comparable infor- 
mation on consumer views of MCO func- 
tioning to complement the more clinically- 
oriented HEDIS measures of health plan 
performance. In 1994, AHCPR sponsored 
the development of the Consumer 
Assessment of Health Plans (CAHPS) sur- 
vey by a consortium of the Harvard 
Medical School, RAND, and the Research 
Triangle Institute 10 . As with HEDIS meas- 
ures, the CAHPS survey was field tested 
and evaluated for validity and reliability. 

It is available for use free of charge. 

The CAHPS survey can be used to assess 
consumer satisfaction with FFS health 
care programs as well as managed care 
plans. Included in CAHPS are versions 
for Medicaid recipients and Medicare 
beneficiaries as well as for persons 
enrolled in employer-based plans, for par- 
ents or guardians to assess services pro- 
vided to children, and for individuals 
with chronic diseases or disabilities. 
CAHPS has a core set of questions appli- 
cable across populations and health care 
delivery systems, and supplemental items 
specific to selected topics. 
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CAHPS is gaining new users rapidly. As of 
early 1999, it was being used by twenty 
states, ten employer groups, by Medicare, 
and by the Federal Employees Health 
Benefits Program, among others. CAHPS 
has been incorporated in HEDIS in lieu of 
the previous NCQA member satisfaction 
measure mentioned above. 

CAHPS survey items tell us what enrollees 
think about their health plans in a variety 
of specific areas. Data analysis follows the 
HEDlS-like formula of A/B =C, generating 
measures of favorable and unfavorable per- 
ceptions concerning health plan functions 
held by a scientifically chosen sample of 
MCO members. Because the data needed 
for analysis is collected via phone or mail 
survey from individuals rather than from 
MCO administrative records or provider 
medical records in accordance with rigid 
protocols, there are fewer data collection 
problems with CAHPS than with HEDIS 
performance measures. This means that 
health care delivery systems with less 
structure and accountability than MCOs 
can also utilize CAHPS. A growing number 
of states are using CAHPS to compare the 
perceptions of Medicaid beneficiaries 
enrolled in MCOs, PCCM, and the FFS 
program. 

CAHPS has many strengths, but it has limi- 
tations as well. The size and representative- 
ness of the sample of members surveyed is 
the key to the validity of responses. Great 
care in survey administration and data 
analysis are essential. Poorly drawn sam- 
ples, samples too small for generalization, 
or carelessness in survey administration 
and analysis can render the results mean- 
ingless, while variation in methods of sur- 
vey administration and analysis can make 
comparison of results impossible. 

Many CAHPS items are designed so 
that they can yield a composite measure- 
ment of a cluster of issues as well as 
scores on discrete items. A series of 
three illustrative questions taken from 
the CAHPS Child Medicaid-Managed 
Care Questionnaire (mail version) is 
shown in Table 3 (page 6). 
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ftiible3: Illustmtive CAHPS Questions* 



With the choices your child’s health plan gave you, how much of a problem, 
if any, was it to get a personal doctor or nurse for your child you are happy with? 



O A big problem O A small problem 
O I didn’t get a new personal doctor or nurse for my child. 



m Promoting Healthy Development of 
children 0-3, a survey of parents con- 
cerning advice, counsel, and assistance 
they have received from their MCO; 

m Adolescent Preventive Care, 14-18, 
which is developing items to be added 
to an adolescent version of CAHPS; and 



Do you have one person you think of as your child’s personal doctor or nurse? 
If you child has more than on personal doctor or nurse, choose the person you 
child seems most often. 



m Living with Illness, a survey of families 
in MCOs with children with chronic or 
special health care needs. 



OYes ONo 

In the last 6 months, when your child went to his or her personal doctor or nurse’s 
office or clinic, how often did the doctor or nurse talk with you about how your 
child is feeling, growing, and behaving? 

O Never O Sometimes O Usually O Always 

O My child doesn’t have a personal doctor or nurse. 



The Foundation for 
Accountability (FACCT) 

B y the mid-1990s, the example of 
HEDIS and the increasing demand 
for information concerning managed 
care performance spawned the 
development of the Foundation for 
Accountability (FACCT) 11 . NCQA grew out 
of purchaser desires to differentiate among 
plan quality and continues to reflect 
assumptions concerning the impact of pur- 
chaser decisions on the health care indus- 
try. FACCT rests on the concept that more 
informed and empowered consumers can 
help shape a health care system which is 
more responsive to individual consumers’ 
needs and desires. 

In keeping with its consumer orientation, 
FACCT has emphasized the organization 
of performance measurement data from 
existing measures into “frameworks” 
which are more readily accessible to con- 
sumers and others outside of the health 
care industry. FACCT’ s framework 
includes the five categories below, and 
has been adopted by NCQA for its own 
HEDIS reporting: 

m “The Basics” - Do people get the basics 
of good care? (e.g., access to care, cus- 
tomer service) 
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M “Staying Healthy” - Are people helped to 
avoid illness and stay as healthy as possi- 
ble? (immunizations, screenings, well-care, 
counseling and anticipatory guidance) 

U “Getting Better” - Are people who are 
sick helped to recover as quickly and 
fully as possible? (early, accurate diag- 
nosis, good treatment and follow-up) 

m “Living With Illness” - Do people get 
help to live as well as possible with ongo- 
ing illness? (good treatment and moni- 
toring, education and counseling, symp- 
tom reduction, patient quality of life) 

m “Changing Needs” - Do people receive 
needed support and care when their 
health or functional abilities change? 
(end of life care, care for people with 
disabilities) 

In addition, FACCT has developed meas- 
urement sets for adult asthma, alcohol mis- 
use, breast cancer, diabetes, major depres- 
sive disorder, health status and health risks, 
some of which compliment HEDIS meas- 
ures related to the same condition. Some of 
these FACCT measures are being utilized in 
the field. It is currendy developing meas- 
ures of child and adolescent health through 
the Child and Adolescent Health 
Measurement Initiative (CAHMI), which it 
has undertaken in conjunction with NCQA 
The measures, which take the form of 
extensive survey instruments, are: 



At present these measures are being field 
tested. 

Table 4 displays an illustrative question 
from the FACCT Promoting Healthy 
Development field trial (page 7). 

How child advocates can use 
performance measurement to 
improve child health 

The development of standardized perform- 
ance measurement, its already widespread 
use in Medicaid, and its broad planned use 
in CHIP programs provide significant 
opportunities for child advocates to improve 
child health services in two broad areas: 

m increasing accountability of MCOs pro- 
viding health care services to children in 
CHIP and Medicaid programs, and 

m driving effective programs of quality 
improvement in both the delivery of 
health care services and in consumer 
service. 

As noted, the underlying concept of per- 
formance measurement is a simple one. It 
has not yet been widely embraced among 
advocates, in part because NCQA contin- 
ues to be oriented toward purchasers, 
while CAHPS and FACCT are oriented 
toward individual consumers. However, 
there are unique opportunities for child 
advocates in the states to play critical roles 
in shaping how performance measure- 
ment in CHIP and Medicaid is structured, 
how it is used, and whether it leads to 
improved health care for children. 

Most state Medicaid and/or CHIP agencies 
(where they are separate) have advisory 
bodies concerned with quality oversight 
matters. In some instances the membership 
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Table 4 : Sample Interview Question front FACC 7 . Healthy Development Field trial 



In the last 12 months, 
did your doctor or 
other health provider 
talk with you about 
the following: 


YES, and I under- 
stand completely. 


YES, and! under- 
stand pretty well. 


YES, but lam 
still unsure. 


NO, we did not 
discuss this but 1 
wish we had. 


NO, we did not 
discuss this but my 
questions were 
answered through 
other resources. 


a a Your child’s growth 1 

jj and development * ... w A 




• .v' /: 






b The kinds of behav- 
ior you can expect to 
see in your child 












c How to dress, bathe, 1 
l and feed your child 












d The importance of 
talking, reading to, 
and playing with 
your child 












I e Things you can do 
|j to help your child 
grow and learn 






lAv A \K 


•j*’*- AUfJry '.. v 2 v 
: .'V .v v :> y i V'.‘ * V dyy .;*• • 




f How to make your 
house and car safe 
for your child 












g Ways to prevent 
your child from j 

! injuries due to i 

falling down 


• ■ 




’ * A A Ay 


• j 


' ’ -V' •VsC 7 ’> ; ■ '■ ’’ 

d!-}; vrf 



is specified in statute or regulation, but in 
many instances memberships are designat- 
ed by the agency or open to interested par- 
ties. These bodies have a variety of titles, 
and vary in formality. What they have in 
common, however, is that they advise on 
key elements of performance measurement 
programs such as: 

H what type of performance measurement 
the state requires; 

m how vigorously the state implements its 
requirements with the affected MCOs; 

H whether there is an advisory group 
bringing the MCOs together with 
other stakeholders; and, perhaps most 
important of all, 

m what uses are made of the resulting 
measures. 







In each of these areas, there is a wide range 
of potential decisions which advocates can 
help shape. Some courses of action lead to 
best health care for kids. Others lead to inef- 
fective quality oversight, or to an emphasis 
on issues affecting state-MCO relations with 
less attention to the impact on enrolled chil- 
dren. If informed child advocates are at the 
table articulating the child-centered alterna- 
tives, the chances of better results increase. 

For example, while performance measure- 
ment is required in both Medicaid and 
CHIP, states have wide discretion in what 
measurements they implement. Some use 
‘home-grown” methods which have not 
been field tested, which do not necessarily 
yield reliable or valid results, and which 
cannot be compared across plans or with 
results from other states. Knowledgeable 



advocates can make the case for the use of 
proven performance method tools - 
HEDIS, CAHPS, or (eventually) FACCT, as 
at least a portion of the state’s perform- 
ance measurement strategy. Further, this 
advisory body is a good venue for discus- 
sion of whether the state’s PCCM and/or 
FFS programs should also be subject to 
performance measurement requirements 
in order to spread the accountability net. 

Whether the state’s program is effectively 
implemented depends in part, as with any 
program, on resource allocation and pri- 
orities. Having knowledgeable child advo- 
cates at the table to reiterate the impor- 
tance of the performance measurement 
program can remind everyone of the need 
to make the effort a priority and to allocate 
resources accordingly. 
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Some states have found it very effective to establish an advisory 
group on performance measurement in which the affected 
MCOs and state agency staff discuss the program, exchange 
information, and solve problems. Child advocates can propose 
the creation of such a group, and explain that they should also 
be included at that table. Their presence will reinforce the 
importance of moving ahead if the MCOs balk (a not unlikely 
event, given the complexity of data collection and stakes for the 
MCOs), and will implicitly remind state staff of the need to keep 
focused on the results for kids. 

Most important of all is the fact that such advisory groups typi- 
cally have significant input into decisions concerning the uses to 
which the performance measurement results will be put. There 
is a wide variety of possibilities. One is reports to legislators, 
and/or to the broader public. Sometimes this is legislatively 
mandated. It is generally an excellent idea, but only after at least 
one year of “practice” by the MCOs. First year results are virtual- 
ly certain to reflect errors in the performance measure collection 
process as much as actual performance, especially where HED1S 
is involved. It would be a public disservice to initiate the public 
reporting program with flawed data. Since it is well known in 
the health care industry that first year data are typically flawed, 
it might also provoke an understandable MCO backlash. (This 
is not to say that first year data should not be reported to the 
state and discussed by the state and MCOs - perhaps even the 
broader oversight committee, if circumstances permit.) 

In the second and subsequent years, data might be used to 
produce “report cards” comparing MCO performance with that 
of other MCOs in the state and perhaps the PCCM program 
and/or FFS programs as well. Such report cards could be used 
in outreach and education programs to parent groups. Further, 
a review of performance measurement results can identify 
those areas in which each MCO has particular opportunities 
for improvement. The review could be used to advocate a plan 
whereby the state would work with each MCO to identify a tar- 
get level of improved performance in one or two key areas to 
be reached within the year or perhaps two. Remeasurement 
should occur, using the same performance measurement tool. 
MCOs with particularly high performance improvement results 
could be rewarded, those who consistently fail could ultimate- 
ly be sanctioned. 



Conclusion 

T o begin, investigate how your state measures Medicaid 
and CHIP MCOs. Identify the advisory and oversight 
bodies, and find out how you can be included. Call on 
NACA’s health staff for assistance. Opportunities to 
make a positive difference for children’s health care are 
waiting for advocates to take advantage of them. 



O 

ERIC 



Endnotes 

1 The tom “managed care” is used here to mean only pre-paid, capitated managed care 
organizations plans - referred to here at “MCOs” - at risk for the cost of a comprehensive 
benefit package. It does not include contracts with plans for limited benefits, like behav- 
ioral health, nor does it include Primary Care Case Management (PCCM) systems. Note 
that in the Medicaid environment, PCCM programs are also technically considered to 
be managed care plans, as PCCM programs assign individual beneficiaries to a primary 
care case “manager” who is responsible for overseeing that individual’s health care and 
for authorizing services. In a PCCM program, however, many or most actual health care 
services are delivered through the fee-for-service (fis) health care delivery system. 

2 An American Public Human Services Association (APHSA) publication, “Monitoring 
the Performance of Managed Care Plans: State Utilization of HEDIS,” (Washington 
Memo, January 1999, pp. 3-8) found that, as of July 1998, 43 states and the District of 
Columbia had a contract with at least one MCO to provide Medicaid services. 

3 1998 State Children’s State Health Program Annual Report, National Conference of 
State Legislatures and the National Governors Association; APHSA web site at 
http://medicaid.uphsa.org/clippage/htm. 

4 Evaluation tools have been produced by other organizations. However, these evalua- 
tion tools do not have the precision, reliability, or standardization of HEDIS, CAHPS, 
and FACCT measures which will be discussed in this Issue Brief, and cannot be used 
for the same kinds of plan comparison, benchmarking, and quality improvement 
measurement As a result, they are not discussed here. 

5 HEDIS and the other managed care performance measurement tools which will be 
discussed in this Issue Brief can be applied to the Medicaid Primary Care Case 
Management (PCCM) program, which enrolls over 4 million members, as well as to 
MCOs. Measurements of consumer attitudes can be applied to some aspects of fee- 
for-service programs as well. 

6 Dr. Wendy Wolf, senior policy fellow, Agency for Health Care Policy and Research 
(AHCPR)/Health Resources and Services Administration (HRSA), presentation to a 
FACCT Task Force meeting, Washington D.C.June 3, 1999 

7 HEDIS is a trademarked product of the National Committee for Quality Assurance 
(NCQA). NCQA is a not-for-profit company with a board of directors including 
providers, health plans, consumers, labor unions, and large employers. NCQA is at 
2000 L Sl, N.W., Washington D.C. 20036. Its web site is www.ncqa.org. The HEDIS 
technical specifications, updated annually, are available from NCQA in HEDIS Volume 
2, which currently costs $245. NACA is working with NCQA to develop an affordable 
method for child advocates to access this technical material. 

8 APHSA report, op. CiL 

9 HRSA Health Resources Services Administration. AHCPR* Agency for Health Care 
Policy Research 

10 Visit the CAHPS web site at www.ahcpr.gov/qual/cahps for information, and to down- 
load the CAHPS instrument and instructions for administration. For additional infor- 
mation, contact Christine Crofton, 301-594-2003, or Charles Darby, 301-594-2050, at 
the Center for Quality Measurement and Improvement, AHCPR. 

11 FACCT is a not-for-profit organization governed by a Board of Trustees including con- 
sumer organizations, corporate purchasers, government purchasers, and providers 
and health plans. Information about the Foundation for Accountability (FACCT) and 
its Child and Adolescent Health Measurement Initiative can be located at the FACCT 
web site, www.FACCT.org, or from FACCT at 520 SW Sixth Avenue, Suite 700, 
Portland OR 97204, 503-223-2228. 
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